Information processing device and method, and program

ABSTRACT

There is provided an information processing device including: a content selection unit configured to select content including at least one name specified by a user; a face group acquisition unit configured to acquire face groups by grouping, per person, face images occurring in content selected by the content selection unit; and a face group determination unit configured to determine a face group to associate with the name from face groups acquired by the face group acquisition unit.

TECHNICAL FIELD

The present disclosure relates to an information processing device,method, and program, and more particularly, to an information processingdevice, method, and program enabling more efficient work of registeringa name and a face image feature value.

BACKGROUND ART

The technology described in Patent Literature 1 may be cited as anexample of related art for personal identification using a face image.Conducting personal identification according to the technology describedin Patent Literature 1 requires advance registration of names and faceimages in association with each other in a catalog. This registrationwork must be conducted manually, and as the number of persons toregister increases, the workload becomes massive.

In contrast, Patent Literature 2 proposes a method of conducting suchassociation between names and face images automatically. The technologydescribed in Patent Literature 2 extracts names from an electronicprogram guide (EPG) of a program recorded by a user, collects multipleprograms in which a specific name occurs, and associates a faceoccurring in common among the collected programs as the facecorresponding to the specific name.

CITATION LIST Patent Literature

Patent Literature 1: JP 2009-53916A

Patent Literature 2: JP 2010-283517A

SUMMARY OF INVENTION Technical Problem

However, since the technology described in Patent Literature 2presupposes that the person appears in the extracted program, in thehypothetical case in which that person's face cannot be detected, thecommon face is lost, and associating names and face images becomesdifficult.

In addition, persons who are not a major character in a program andpersons who appear on a news program may not necessarily be listed inthe EPG. Consequently, with the technology described in PatentLiterature 2, associating names and face images is difficult.

The present disclosure has been devised in light of such circumstances,and enables more efficient work of registering a name and a face imagefeature value.

Solution to Problem

An information processing device according to an aspect of the presentdisclosure includes: a content selection unit configured to selectcontent including at least one name specified by a user; a face groupacquisition unit configured to acquire face groups by grouping, perperson, face images occurring in content selected by the contentselection unit; and a face group determination unit configured todetermine a face group to associate with the name from face groupsacquired by the face group acquisition unit.

The content selection unit may acquire a name occurrence patternindicating whether or not the name occurs within selected content, theface group acquisition unit may acquire face group occurrence patternsindicating whether or not there is an occurrence in all content selectedby the content selection unit, and the face group determination unit maydetermine a face group to associate with the name on the basis of asimilarity between the name occurrence pattern acquired by the contentselection unit, and the face group occurrence patterns acquired by theface group acquisition unit.

The content selection unit may acquire the name occurrence pattern onthe basis of text information or speech information within selectedcontent, or specific person occurrence frequency data in which anoccurrence frequency of a specific person obtained as a result ofidentifying metadata attached to content is expressed in a time series.

A display control unit configured to control display of a screenenabling selection of one face group from a plurality of face groups forwhich the similarity is high, and a selection unit configured to selectthe one face group according to a user operation may be furtherincluded, and the face group determination unit may determine a facegroup selected by the selection unit as the face group to associate withthe name.

A comparison unit configured to compare a face group for which thesimilarity is high with external search result data for the name may befurther included, and the face group determination unit may determine aface group that is most similar to the external search result data fromamong a plurality of face groups for which the similarity is high as theface group to associate with the name.

The content selection unit may select content in which the name possiblyoccurs, the face group acquisition unit may acquire, from among facegroups in each piece of content selected by the content selection unit,as candidate face groups, face groups having a greatest number ofsimilar face groups appearing in other content, and the face groupdetermination unit may cluster candidate face groups acquired by theface group acquisition unit, and determine a face group belonging to acluster with the most candidate face groups as the face group toassociate with the name.

The content selection unit may select content in which the name possiblyoccurs on the basis of text information or speech information withinselected content, or specific person occurrence frequency data in whichan occurrence frequency of a specific person obtained as a result ofidentifying metadata attached to content is expressed in a time series.

The face group acquisition unit may exclude face groups having a lowpossibility to associate with the name from among face groups in eachpiece of content selected by the content selection unit, and acquire,from among the other face groups, as candidate face groups, face groupshaving a greatest number of similar face groups appearing in othercontent.

A display control unit configured to control display of a screenenabling selection of one face group from a plurality of face groupsbelonging to clusters with many of the candidate face groups, and aselection unit configured to select the one face group according to auser operation may be further included, and the face group determinationunit may determine a face group selected by the selection unit as theface group to associate with the name.

A comparison unit configured to compare a plurality of face groupsbelonging to clusters with many of the candidate face groups withexternal search result data for the name may be further included, andthe face group determination unit may determine a face group that ismost similar to the external search result data from among a pluralityof face groups belonging to clusters with many of the candidate facegroups as the face group to associate with the name.

A face image selection unit configured to select a face image toregister in a catalog from among a face image collection in a face groupdetermined by the face group determination unit may be further included.

The face image selection unit may select a representative face in eachcluster as a result of clustering from among a face image collection ina face group determined by the face group determination unit as a faceimage to register in a catalog.

An information processing method according to an aspect of the presentdisclosure, performed by an information processing device, includes:selecting content including at least one name specified by a user;acquiring face groups by grouping, per person, face images occurring inthe selected content; and determining a face group to associate with thename from the acquired face groups.

A program according to an aspect of the present disclosure causes acomputer to function as: a content selection unit configured to selectcontent including at least one name specified by a user; a face groupacquisition unit configured to acquire face groups by grouping, perperson, face images occurring in content selected by the contentselection unit; and a face group determination unit configured todetermine a face group to associate with the name from face groupsacquired by the face group acquisition unit.

In an aspect of the present disclosure, content including at least onename specified by a user is selected, face images occurring in theselected content are grouped per person, and face groups are acquired.Subsequently, a face group to associate with the name is decided fromamong the acquired face groups.

Advantageous Effects of Invention

According to the present disclosure, more efficient work of registeringa name and a face image feature value is possible.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an exemplary configuration of aninformation processing device applying the present technology.

FIG. 2 is a flowchart describing a face catalog registration process ofan information processing device.

FIG. 3 is a diagram illustrating an example of a name occurrence patternand face group occurrence patterns.

FIG. 4 is a flowchart describing a content selection process.

FIG. 5 is a diagram illustrating an example of specific personoccurrence frequency data.

FIG. 6 is a flowchart describing another example of a content selectionprocess.

FIG. 7 is a diagram illustrating an example of person occurrencefrequency data.

FIG. 8 is a diagram illustrating a data example of a name occurrencepattern.

FIG. 9 is a flowchart describing a face group acquisition process.

FIG. 10 is a flowchart describing a face grouping process.

FIG. 11 is a flowchart describing a face group determination process.

FIG. 12 is a block diagram illustrating an exemplary configuration of aface group determination unit.

FIG. 13 is a flowchart describing another example of a face groupdetermination process.

FIG. 14 is a block diagram illustrating another exemplary configurationof a face group determination unit.

FIG. 15 is a flowchart describing yet another example of a face groupdetermination process.

FIG. 16 is a block diagram illustrating another exemplary configurationof an information processing device applying the present technology.

FIG. 17 is a flowchart describing a face catalog registration process ofan information processing device.

FIG. 18 is a flowchart describing a content selection process.

FIG. 19 is a flowchart describing another example of a content selectionprocess.

FIG. 20 is a block diagram illustrating an exemplary configuration of aface group acquisition unit.

FIG. 21 is a flowchart describing a face group acquisition process.

FIG. 22 is a flowchart describing an in-content face group acquisitionprocess.

FIG. 23 is a flowchart describing a candidate face group acquisitionprocess.

FIG. 24 is a diagram describing a candidate face group acquisitionprocess.

FIG. 25 is a block diagram illustrating an exemplary configuration of aface group determination unit.

FIG. 26 is a flowchart describing a face group determination process.

FIG. 27 is a block diagram illustrating another exemplary configurationof a face group determination unit.

FIG. 28 is a flowchart describing another example of a face groupdetermination process.

FIG. 29 is a block diagram illustrating an exemplary configuration of acomputer.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments for carrying out the present disclosure(hereinafter designated embodiments) will be described. Hereinafter, thedescription will proceed in the following order.

1. Embodiment (information processing device)2. Embodiment (information processing device)3. Embodiment (computer)

1. Embodiment Information Processing Device Configuration of InformationProcessing Device According to Present Technology

FIG. 1 is a diagram illustrating an exemplary configuration of aninformation processing device applying the present technology.

As an example, the information processing device 11 in FIG. 1 registersin a catalog a face image and a face image feature value correspondingto a name, and conducts searches on the catalog, even if the input nameis not registered in the catalog. The information processing device 11is made up of a personal computer or the like, for example.

In the example of FIG. 1, the information processing device 11 includesa name input unit 21, a content archive 22, a content selection unit 23,a face group acquisition unit 24, a face group determination unit 25, aregistration face image selection unit 26, and a face catalog 27.

The name input unit 21 inputs a name specified by a user (hereinafteralso designated the specific name), and supplies the input name to thecontent selection unit 23. The content archive 22 registers and managescontent.

The content selection unit 23 selects arbitrary content from inside thecontent archive 22. In addition, the content selection unit 23 acquiresa name occurrence pattern, which is information indicating whether ornot the specific name from the name input unit 21, or in other words theperson having the specific name (hereinafter also designated thespecific person), occurs within each piece of content. Note that herein,at least one piece of content in which the specific person possiblyappears must be included among the selected content. The contentselection unit 23 supplies the acquired name occurrence pattern,together with the selected content, to the face group acquisition unit24. In addition, the content selection unit 23 supplies the acquiredname occurrence pattern to the face group determination unit 25.

The face group acquisition unit 24 conducts a process that groups facesoccurring in each piece of selected content, and collects the sameperson into a single group. In addition, the face group acquisition unit24 acquires for each face group a face group occurrence pattern, whichis information indicating whether or not that face group occurs in eachpiece of content. The face group acquisition unit 24 supplies theacquired face group occurrence pattern to the face group determinationunit 25.

The face group determination unit 25 determines a face group toassociate with the specific name from among the face groups acquired bythe face group acquisition unit 24. Specifically, the face groupdetermination unit 25 conducts a process of comparing the nameoccurrence pattern from the content selection unit 23 with each facegroup occurrence pattern from the face group acquisition unit 24.Subsequently, the face group determination unit 25 determines the facegroup having the face group occurrence pattern that is most similar tothe name occurrence pattern as a face group to associate with thespecific name. The face group determination unit 25 supplies informationabout the determined face group to the registration face image selectionunit 26.

The registration face image selection unit 26 selects and registers inthe face catalog 27 a specified number of face images and face imagefeatures values from among all face images belonging to all face groupsdetermined by the face group determination unit 25. The face images tobe selected may be representative faces from each face group acquired bythe face group determination unit 25, or all faces belonging to eachface group.

For example, when using representative faces from each face group, therepresentative face images from each face group are categorized into anarbitrary number of clusters according to face feature value. For theclustering, an arbitrary technique such as k-means clustering is used.The registration face image selection unit 26 acquires a representativeface in each cluster, and registers the representative image or facefeature value from each cluster in the face catalog 27. Alternatively,categorization by the time period when content was recorded, the contenttype, and the like is also possible.

The face catalog 27 registers and manages face images and face imagefeature values in association with names.

Operation of Information Processing Device

Next, a face catalog registration process which is an operation of theinformation processing device 11 will be described with reference to theflowchart in FIG. 2.

For example, a name specified by the user is input into the contentselection unit 23 via the name input unit 21. In step S11, the contentselection unit 23 conducts a content selection process. The contentselection process will be described later in detail with reference toFIG. 4.

According to the process in step S11, there is acquired a nameoccurrence pattern, which is information indicating whether or not thespecific name occurs in each piece of content. The name occurrencepattern is supplied together with the content to the face groupacquisition unit 24.

In step S12, the face group acquisition unit 24 conducts a face groupacquisition process. The face group acquisition process will bedescribed later in detail with reference to FIG. 9.

According to the process in step S12, faces occurring in each piece ofselected content are grouped, and for each group, there is acquired aface group occurrence pattern, which is information indicating whetheror not that face group occurs in each piece of content. The face groupoccurrence patterns are supplied to the face group determination unit25.

In addition, the name occurrence pattern acquired in step S11 is alsosupplied to the face group determination unit 25.

In step S13, the face group determination unit 25 conducts a face groupdetermination process. The face group determination process will bedescribed later in detail with reference to FIG. 12.

According to the process in step S13, a process of comparing the nameoccurrence pattern from the content selection unit 23 and each facegroup occurrence pattern from the face group acquisition unit 24 isconducted. Subsequently, the face group having the face group occurrencepattern that is most similar to the name occurrence pattern isdetermined as the face group to associate with the name.

FIG. 3 illustrates an example of a name occurrence pattern forrespective pieces of content, and respective face group occurrencepatterns for face groups 1 to 3. In the example of FIG. 3, “1” indicatesthat the name occurs in the content, while “0” indicates that the namedoes not occur.

Since the input name occurs in the content A, but does not occur in thecontent B and C, the name occurrence pattern becomes “100”. Since theface group 1 does not occur in the content A and C, but occurs in thecontent B, the group occurrence pattern of the face group 1 becomes“010”. Since the face group 2 occurs in the content A and C, but doesnot occur in the content B, the group occurrence pattern of the facegroup 2 becomes “101”. Since the face group 3 occurs in the content A,but does not occur in the content B and C, the group occurrence patternof the face group 3 becomes “100”.

Consequently, the face group having the group occurrence pattern that ismost similar to the name occurrence pattern “100” is the face group 3,and the face group 3 is determined as the face group to associate withthe person. Information about the determined face group is supplied tothe registration face image selection unit 26.

In step S14, the registration face image selection unit 26 conducts aregistration face image selection process. In other words, theregistration face image selection unit 26 selects a specified number offace images and face image feature values from among the face groupdetermined by the face group determination unit 25, and registers in theface catalog 27 the selected face images and face image feature valuesin association with the name.

According to the above, in the information processing device 11, faceimages and face image feature values are registered in the face catalog27 in association with a name, enabling searches to be conducted.Consequently, more efficient catalog registration work is possible.

Example of Content Selection Process

Next, the content selection process in step S11 of FIG. 2 will bedescribed with reference to the flowchart in FIG. 4. In the contentselection process, an arbitrary number of pieces of content are acquiredfrom inside the content archive 22, so that at least one piece ofcontent in which the specific person possibly appears is included.

The content selection unit 23 acquires one piece of content from thearchive (content archive 22) in step S31, and selects a frame in stepS32.

In step S33, the content selection unit 23 conducts variousidentification processing on the selected frame. In other words, insteps S32 and S33, at an arbitrary time interval, metadata attached tothe content is read, and the various identification processing judgeswhether or not the specific name or speech by the specific personoccurs.

Herein, the identification processing refers to processing to identifytext information in a screen, and may be processing to identify spokencontent, and additionally is processing using text, speech, or otherinformation in a picture, such as processing to identify a speaker.Consequently, the identification processing is not required to belimited to the above, insofar as it is possible to identify whether ornot the specific person is present.

In step S34, every time the above identification processing isconducted, the content selection unit 23 updates specific personoccurrence frequency data as a result. The details written to thespecific person occurrence frequency data are a time position at whichidentification was conducted, and information indicating whether or notthere is an occurrence. Herein, the information indicating whether ornot there is an occurrence may be included for every identificationprocessing result, collected into a single piece of information, orboth. In addition, information such as the size of the text and thevolume of speech at the time of identification may also be written.

FIG. 5 illustrates an example of specific person occurrence frequencydata. In the example of FIG. 5, a text identification result, a spokencontent identification result, and a full identification result withrespect to the specific person identified every five minutes, as well asan identification result indicating the presence of metadata, areindicated in a time series.

The example of FIG. 5 indicates that 5 minutes after the start of thecontent, there was a text identification result with respect to thespecific person. Also indicated is that 5 minutes after and 10 minutesafter the start of the content, there was a speech identification resultwith respect to the specific person. Also indicated is that 15 minutesafter and 20 minutes after the start of the content, there was a speechidentification result with respect to the specific person.

Additionally, it is indicated that from 5 minutes to 20 minutes afterthe start of the content, there was a full identification result.Furthermore, it is indicated that from 5 minutes to 20 minutes after thestart of the content, there was metadata.

Returning to FIG. 4, in step S35, the content selection unit 23determines whether or not all frames have been processed. In the case ofdetermining that not all frames have been processed in step S35, theprocess returns to step S32, and the processing thereafter is repeated.

In step S35, in the case of determining that all frames have beenprocessed, the process proceeds to step S36. In step S36, the contentselection unit 23 updates the name occurrence pattern on the basis ofthe updated specific person occurrence frequency data.

In step S37, the content selection unit 23 determines whether or not aspecified number of pieces of content have been acquired. In step S37,in the case of determining that the specified number of pieces ofcontent have not been acquired, the process returns to step S31, and theprocessing thereafter is repeated.

In step S37, in the case of determining that the specified number ofpieces of content have been acquired, the process proceeds to step S38.In step S38, the content selection unit 23 determines whether or not thenumber of pieces of content in which the specific person appears is 0.In the case of determining that the number of pieces of content in whichthe specific person appears is 0 in step S38, the process proceeds tostep S39.

In step S39, the content selection unit 23 removes one piece of content,and the process returns to step S31. In step S38, in the case ofdetermining that the number of pieces of content in which the specificperson appears is not 0, the content selection process in FIG. 4 ends.

Note that an example is described in which the identification processingin the content selection process described above is conducted every timea name is specified. However, the identification processing is notrequired to be conducted every time a name is specified. In other words,information related to a person occurring in a piece of content and thefrequency of occurrence may be acquired in advance for each piece ofcontent and stored in memory or the like, for example, and on the basisof the information, the name occurrence pattern may be updated. Anexample of the content selection process in this case is illustrated inthe following FIG. 6.

Another Example of Content Selection Process

Next, another example of the content selection process in step SI 1 ofFIG. 2 will be described with reference to the flowchart in FIG. 6.

In step S51, the content selection unit 23 acquires one piece of contentfrom the archive (content archive 22). In step S52, the contentselection unit 23 acquires name occurrence frequency data for theacquired content from memory (not illustrated) or the like, for example.

FIG. 7 illustrates an example of person occurrence frequency data. Inthe example of FIG. 7, a text identification result, a spoken contentidentification result, and a full identification result with respect tomultiple persons occurring in the content (for example, James and Mary)identified every five minutes, as well as an identification resultindicating the presence of metadata, are indicated in a time series.

The example of FIG. 7 indicates that 5 minutes after the start of thecontent, there was a text identification result for James, while 15minutes after and 20 minutes after, there was a text identificationresult for Mary. Also indicated is that 5 minutes after and 10 minutesafter the start of the content, there was a speech identification resultfor James, while 15 minutes after and 20 minutes after, there was aspeech identification result for Mary. Also indicated is that 15 minutesafter and 20 minutes after the start of the content, there was a speechidentification result for James, while 25 minutes after, there was aspeech identification result for Mary.

Additionally, it is indicated that from 5 minutes to 20 minutes afterthe start of the content, there was a full identification result forJames, while from 15 minutes to 25 minutes after, there was a fullidentification result for Mary. Furthermore, it is indicated that from 5minutes to 20 minutes after the start of the content, there was metadatafor James, while 25 minutes after, there was metadata for Mary.

Returning to FIG. 6, in step S53, the content selection unit 23 updatesthe name occurrence pattern on the basis of the acquired personoccurrence frequency data.

In step S54, the content selection unit 23 determines whether or not aspecified number of pieces of content have been acquired. In step S54,in the case of determining that the specified number of pieces ofcontent have not been acquired, the process returns to step S51, and theprocessing thereafter is repeated.

In step S54, in the case of determining that the specified number ofpieces of content have been acquired, the process proceeds to step S55.In step S55, the content selection unit 23 determines whether or not thenumber of pieces of content in which the specific person appears is 0.In the case of determining that the number of pieces of content in whichthe specific person appears is 0 in step S55, the process proceeds tostep S56.

In step S56, the content selection unit 23 removes one piece of content,and the process returns to step S51. In step S55, in the case ofdetermining that the number of pieces of content in which the specificperson appears is not 0, the content selection process in FIG. 6 ends.

Data Example of Name Occurrence Pattern

FIG. 8 is a diagram illustrating a data example of a name occurrencepattern. The data of the name occurrence pattern is configured so that aspecific person occurs if the occurrence frequency is greater than athreshold value.

For example, in FIG. 8A, “1” indicates that the specific person occursin the content, while “0” indicates that the specific person does notoccur. Since the name occurs in the content A, but does not occur in thecontent B and C, the person occurrence pattern becomes “100”.

In this way, the data of the name occurrence pattern may be expressedwith the two-level values of 1 and 0.

FIG. 8B illustrates an example of using n-level values to express thedegree of occurrence frequency with respect to the content overall asthe data of the name occurrence pattern. In the case of FIG. 8B, thecontent A is “60”, the content B is “5”, and the content C is “1”. Thismeans that the specific person occurs at a ratio of 60 in the content A,occurs at a ratio of 5 in the content B, and does not occur in thecontent C.

Note that the examples of FIG. 8A and FIG. 8B illustrate examples ofexpressing one value for one piece of content. In contrast, asillustrated in FIG. C, content may be subdivided into several sections,a value related to occurrence may be computed for each section, and thevalues may be combined to express the data of the name occurrencepattern. In other words, it is possible to treat information related tooccurrence frequency as a number of n-level values equal to the numberof sections.

In FIG. 8C, the data of the name occurrence pattern of the content A isindicated as “90-20-70”, while the data of the name occurrence patternof the content B is indicated as “5-0-10”. Also, the data of the nameoccurrence pattern of the content C is indicated as “0-0-0”.

In other words, in the case of FIG. 8C, it is indicated that, in thecontent A, the specific person occurs at a ratio of “90” in a firstsection, occurs at a ratio of “20” in a second section, and occurs at aratio of “70” in a third section. Also indicated is that, in the contentB, the specific person occurs at a ratio of “5” in a first section, doesnot occur in a second section, and occurs at a ratio of “10” in a thirdsection. Also indicated is that, in the content C, the specific persondoes not occur in any of a first section to a third section.

Herein, the information related to occurrence frequency may be computedby using a collection of full identification results as described above,or by using only identification results specified by the user. Inaddition, not only the occurrence frequency but also the text size orvolume may be used to apply a weighting.

Example of Face Group Acquisition Process

Next, the face group acquisition process in step S12 of FIG. 2 will bedescribed with reference to the flowchart in FIG. 9. In this process,face images occurring in all content selected by the content selectionunit 23 are detected, a grouping process is conducted per person, andonly face groups having a high likelihood of correspondence with thespecific person are acquired.

In step S71, the face group acquisition unit 24 conducts a face groupingprocess. Note that the face grouping process will be described later indetail with reference to FIG. 10.

According to the process in step S71, face images occurring within thecontent are detected, the detected face images are grouped per person,and the face groups are registered in a face group list, which is a listof face groups. Note that this process is conducted on all contentselected by the content selection unit 23. In addition, grouping is notconducted per piece of content, but instead conducted over all content.

In step S72, the face group acquisition unit 24 selects one face groupfrom the face group list, and conducts personal identification per facegroup. In step S73, the face group acquisition unit 24 determineswhether or not a face belonging to the face group selected in step S72matches a person already registered in a built-in catalog.

In step S73, in the case of determining that a face belonging to theselected face group matches an already-registered person, the processproceeds to step S74. In other words, in this case, since the face groupis clearly not of a face that should be associated with the specificperson input into the name input unit 21, in step S74, the face groupacquisition unit 24 removes that face group from the face group list.

In step S73, in the case of determining that a face belonging to theselected face group does not match an already-registered person, theprocess proceeds to step S75. In step S75, the face group acquisitionunit 24 creates a face group occurrence pattern for that face group.

This pattern data is created using the same criteria as when creatingthe name occurrence pattern, in other words, the same criteria as whencreating the name occurrence pattern, that is, the same number of levelsn of a variable indicating the occurrence ratio, the same number ofsections subdividing the content, and the same section divisionpositions. Values related to the occurrence ratio may be computedaccording to a method similar to the name occurrence pattern, but mayalso be computed by using weighted factors in addition to the occurrenceratio of the face image, such as the size of the face image, theposition of the face (distance from screen center), and the number ofpersons occurring at the same time.

In step S76, the face group acquisition unit 24 determines whether ornot the above process has been conducted on all face groups. In stepS76, in the case of determining that not all face groups have beenprocessed, the process returns to step S72, and the processingthereafter is repeated.

In step S76, in the case of determining that all face groups have beenprocessed, the face group acquisition process of FIG. 9 ends.Subsequently, the face group occurrence patterns created for every facegroup are supplied to the face group determination unit 25.

Example of Face Grouping Process

Next, the face grouping process in step S11 of FIG. 9 will be describedwith reference to the flowchart in FIG. 10. Note that this process isconducted on the entirety of a moving image at an arbitrary timeinterval starting from the first frame of the moving image.

In step S91, the face group acquisition unit 24 determines whether ornot a face image has been detected. In step S91, in the case ofdetermining that a face image has been detected, the process proceeds tostep S92.

In step S92, the face group acquisition unit 24 determines whether ornot the current number of groups is greater than 0. In step S92, in thecase of determining that the current number of groups is 1 or greater,the process proceeds to step S93.

In step S93, the face group acquisition unit 24 conducts a similarityevaluation for each group. In other words, the face group acquisitionunit 24 evaluates the similarity of face images registered in anexisting group to the face image that was just detected.

In step S94, the face group acquisition unit 24 determines whether ornot the greatest similarity computed from among the groups (maximumsimilarity) is greater than a threshold value. In step S94, in the caseof determining that the maximum similarity is greater than the thresholdvalue, the process proceeds to step S95.

In step S95, the face group acquisition unit 24 adds the detected faceimage to the group with the maximum similarity. In other words, the facegroup acquisition unit 24 treats the detected face image as a face ofthe same person as the faces registered in the group that yielded themaximum similarity, and adds the detected face image as a member of thatgroup.

On the other hand, in step S92, in the case of determining that thecurrent number of groups is 0, the process proceeds to step S96. Also,in step S94, in the case of determining that the maximum similarity isless than or equal to the threshold value, the detected face image islikewise treated as a separate person from any of the groups, and theprocess proceeds to step S96. In step S96, the face group acquisitionunit 24 generates a new face group, and adds the detected face image asa member. Subsequently, the face group acquisition unit 24 registers thecreated face group in the face group list.

In step S91, in the case of determining that a face image has not beendetected, the process proceeds to step S97. In step S97, the face groupacquisition unit 24 determines whether or not all frames constitutingthe moving image have been processed.

In step S97, in the case of determining that not all frames have beenprocessed, the process returns to step S91, and the processingthereafter is repeated on a frame at an arbitrary time interval. In stepS97, in the case of determining that all frames have been processed, theface grouping process ends, and the process returns to step S71 in FIG.9.

Note that the face grouping process is not limited to the processdescribed with reference to FIG. 10, and that any method may be usedinsofar as grouping is possible.

Example of Face Group Determination Process

Next, the face group determination process in step S13 of FIG. 2 will bedescribed with reference to the flowchart in FIG. 11. In this process,similarity is evaluated between the name occurrence pattern computed bythe content selection unit 23 and face group occurrence patternscomputed by the face group acquisition unit 24, and on the basis of theresults, a face group to associate with the specific name is determined.

In step S111, the face group determination unit 25 loads a nameoccurrence pattern from the content selection unit 23. In step S112, theface group determination unit 25 acquires the distance of similaritybetween the loaded name occurrence pattern and a face group occurrencepattern acquired by the face group acquisition unit 24.

In step S113, the face group determination unit 25 determines whether ornot the processing in step S112 has been conducted on all face groups.In step S113, in the case of determining that not all face groups havebeen processed, the process returns to step S112, and the processingthereafter is repeated.

In step S113, in the case of determining that all face groups have beenprocessed, the process proceeds to step S114. In step S114, the facegroup determination unit 25 associates the group of minimum distancewith the specific name.

Note that although the above describes an example of selecting the facegroup having the minimum distance when deciding a face group, asdescribed next, several high-ranking face groups may also be treated ascandidates and presented to the user for selection by the user.

Exemplary Configuration of Face Group Determination Unit

FIG. 12 is a block diagram illustrating an exemplary configuration of aface group determination unit in the case in which several high-rankingface groups are treated as candidates and presented to the user forselection by the user.

In the example of FIG. 12, the face group determination unit 25 includesa name occurrence pattern input unit 131, a face group occurrencepattern input unit 132, a name/face group distance acquisition unit 133,a selection screen display control unit 134, and a face groupconfiguration unit 135.

Also, in this case, the information processing device 11 also includes adisplay unit 141 and an operating input unit 142 in addition to theconfiguration discussed earlier with reference to FIG. 1.

The name occurrence pattern input unit 131 loads and supplies to thename/face group distance acquisition unit 133 a name occurrence patternfrom the content selection unit 23. The face group occurrence patterninput unit 132 loads and supplies to the name/face group distanceacquisition unit 133 face group occurrence patterns from the face groupacquisition unit 24.

The name/face group distance acquisition unit 133 acquires the distancebetween the name occurrence pattern and a face group occurrence pattern,for all face groups. Subsequently, the name/face group distanceacquisition unit 133 treats the face group of minimum distance as afirst candidate face group, and supplies information about severalhigh-ranking face groups to the selection screen display control unit134 and the face group configuration unit 135.

In one process, when there is a user selection, the selection screendisplay control unit 134 generates a selection screen enabling theselection of one face group name from among candidate face group namesmade up of face group names that are candidates for the face group toassociate with the name. The selection screen display control unit 134causes the display unit 141 to display the generated selection screen.In addition, on the basis of a face group selection signal by the userinput via the operating input unit 142, the selection screen displaycontrol unit 134 treats the user-selected face group as a firstcandidate face group, and supplies information about severalhigh-ranking face groups to the face group configuration unit 135.

In one process, when there is a user selection, the face groupconfiguration unit 135 configures the first candidate face groupsupplied from the selection screen display control unit 134 as the facegroup to associate with the specific name. In one process, when there isno user selection, the face group configuration unit 135 configures thefirst candidate face group supplied from the name/face group distanceacquisition unit 133 as the face group to associate with the specificname.

The display unit 141 is made up of a liquid crystal display (LCD) or thelike, for example, and displays a selection screen from the selectionscreen display control unit 134.

The operating input unit 142 is made up of a mouse and keyboard or atouch panel stacked onto or under the display unit 141, for example, andsupplies a signal corresponding to a user operation to the selectionscreen display control unit 134. For example, a selection signal of aface group on the selection screen is supplied to the face groupconfiguration unit 135 and the like via the selection screen displaycontrol unit 134.

Example of Face Group Determination Process

Next, the face group determination process in step S13 of FIG. 2executed by the face group determination unit 25 of FIG. 12 will bedescribed with reference to the flowchart in FIG. 13.

In step S131, the name occurrence pattern input unit 131 loads andsupplies to the name/face group distance acquisition unit 133 a nameoccurrence pattern from the content selection unit 23. At this point,the face group occurrence pattern input unit 132 loads and supplies tothe name/face group distance acquisition unit 133 face group occurrencepatterns from the face group acquisition unit 24.

In step S132, the name/face group distance acquisition unit 133 acquiresthe distance of similarity between the name occurrence pattern from thename occurrence pattern input unit 131 and a face group occurrencepattern from the face group occurrence pattern input unit 132.

In step S133, the name/face group distance acquisition unit 133determines whether or not all face groups have been processed. In thecase of determining that not all face groups have been processed, theprocess returns to step S132 in step S133, and the processing thereafteris repeated.

In the case of determining that all face groups have been processed instep S133, the process proceeds to step S134. In step S134, thename/face group distance acquisition unit 133 treats the face group ofminimum distance as a first candidate face group, and suppliesinformation about several high-ranking face groups to the selectionscreen display control unit 134 and the face group configuration unit135.

In step S135, in one process, the selection screen display control unit134 determines whether or not there is a user selection. In the case ofdetermining that there is a user selection in step S135, the processproceeds to step S136.

In step S136, the selection screen display control unit 134 generates aselection screen enabling the selection of one face group name fromamong candidate face group names made up of face group names that arecandidates for the face group to associate with the name. Subsequently,the selection screen display control unit 134 causes the display unit141 to display the generated selection screen.

The user operates the operating input unit 142 to select a face groupname to associate with the name. The operating input unit 142 supplies aselection signal corresponding to the selection to the selection screendisplay control unit 134.

In step S137, the selection screen display control unit 134 acquires aselection result from the operating input unit 142. In step S138, theselection screen display control unit 134 treats the user-selected facegroup as a first candidate face group, and supplies information aboutseveral high-ranking face groups to the face group configuration unit135.

On the other hand, in step S135, in the case of determining that thereis no user selection, the process skips steps S136 to S138, and proceedsto step S139.

In step S139, the face group configuration unit 135 configures a facegroup. In other words, in one process, when there is a user selection,the face group configuration unit 135 configures the first candidateface group supplied from the selection screen display control unit 134as the face group to associate with the specific name. In one process,when there is no user selection, the face group configuration unit 135configures the first candidate face group supplied from the name/facegroup distance acquisition unit 133 as the face group to associate withthe specific name.

Note that although the above describes an example of determining a facegroup in which several high-ranking face groups are treated ascandidates and presented to the user for selection by the user, asdescribed next, a selection may also be made using a network service orthe like as external data.

Exemplary Configuration of Face Group Determination Unit

FIG. 14 is a block diagram illustrating an exemplary configuration of aface group determination unit that makes a selection using a networkservice or the like as external data.

In the example of FIG. 14, the face group determination unit 25 includesa name occurrence pattern input unit 131, a face group occurrencepattern input unit 132, and a name/face group distance acquisition unit133. Additionally, the face group determination unit 25 includes anexternal data input unit 151, an external data comparison unit 152, aselection screen display control unit 153, and a face groupconfiguration unit 154.

The face group determination unit 25 of FIG. 14 is equipped with a nameoccurrence pattern input unit 131, a face group occurrence pattern inputunit 132, and a name/face group distance acquisition unit 133, in commonwith the face group determination unit 25 of FIG. 12.

The face group determination unit 25 of FIG. 14 differs from the facegroup determination unit 25 of FIG. 12 in that the selection screendisplay control unit 134 and the face group configuration unit 135 arereplaced with a selection screen display control unit 153 and a facegroup configuration unit 154. In addition, the face group determinationunit 25 of FIG. 14 differs from the face group determination unit 25 ofFIG. 12 with the addition of an external data input unit 151 and anexternal data comparison unit 152.

In other words, the name/face group distance acquisition unit 133 treatsthe face group of minimum distance as a first candidate face group, andsupplies information about several high-ranking face groups to theexternal data comparison unit 152, the selection screen display controlunit 153, and the face group configuration unit 154.

The external data input unit 151 inputs and supplies to the externaldata comparison unit 152 external data made up of face feature valuesand the like of several high-ranking image search results conducted byan external network service or the like for the same name as the oneinput into the name input unit 21 of FIG. 1.

The external data comparison unit 152 compares the face feature valuesof several high-ranking image search results conducted by an externalnetwork service or the like from the external data input unit 151 withrepresentative face feature values of several high-ranking face groupsof small distance from the name/face group distance acquisition unit133. As a result of the comparison, the external data comparison unit152 treats the face group with the highest degree of similarity with theexternal data as a first candidate face group, and supplies informationabout several high-ranking face groups to the selection screen displaycontrol unit 153 and the face group configuration unit 154.

In one process, when external data is used and there is a userselection, the selection screen display control unit 153 generates aselection screen made up of face group names and the like that arecandidates for the face group to associate with the name, on the basisof the face group information from the external data comparison unit152. In one process, when external data is not used and there is a userselection, the selection screen display control unit 153 generates aselection screen made up of face group names and the like that arecandidates for the face group to associate with the name, on the basisof the face group information from the name/face group distanceacquisition unit 133. The selection screen display control unit 153causes the display unit 141 to display the generated selection screen.

On the basis of a face group selection signal by the user input via theoperating input unit 142, the selection screen display control unit 153treats the user-selected face group as a first candidate face group, andsupplies information about several high-ranking face groups to the facegroup configuration unit 135.

In one process, when there is a user selection, the face groupconfiguration unit 154 configures the first candidate face groupsupplied from the selection screen display control unit 153 as the facegroup to associate with the specific name. In one process, when externaldata is used and there is no user selection, the face groupconfiguration unit 154 configures the first candidate face groupsupplied from the external data comparison unit 152 as the face group toassociate with the specific name. In one process, when external data isnot used and there is no user selection, the face group configurationunit 154 configures the first candidate face group supplied from thename/face group distance acquisition unit 133 as the face group toassociate with the specific name.

Another Example of Face Group Determination Process

Next, the face group determination process in step S13 of FIG. 2executed by the face group determination unit 25 of FIG. 14 will bedescribed with reference to the flowchart in FIG. 15.

In step S151, the name occurrence pattern input unit 131 loads andsupplies to the name/face group distance acquisition unit 133 a nameoccurrence pattern from the content selection unit 23. At this point,the face group occurrence pattern input unit 132 loads and supplies tothe name/face group distance acquisition unit 133 face group occurrencepatterns from the face group acquisition unit 24.

In step S152, the name/face group distance acquisition unit 133 acquiresthe distance of similarity between the name occurrence pattern from thename occurrence pattern input unit 131 and a face group occurrencepattern from the face group occurrence pattern input unit 132.

In step S153, the name/face group distance acquisition unit 133determines whether or not all face groups have been processed. In thecase of determining that not all face groups have been processed in stepS153, the process returns to step S152, and the processing thereafter isrepeated.

In the case of determining that all face groups have been processed instep S153, the process proceeds to step S154. In step S154, thename/face group distance acquisition unit 133 treats the face group ofminimum distance as a first candidate face group, and suppliesinformation about several high-ranking face groups to the external datacomparison unit 152, the selection screen display control unit 153, andthe face group configuration unit 154.

In step S155, in one process, the external data comparison unit 152determines whether or not to use external data. In the case ofdetermining to use external data in step S155, the process proceeds tostep S156.

In step S156, the external data comparison unit 152 conducts a processof comparing several high-ranking face groups of small distance from thename/face group distance acquisition unit 133 with external data fromthe external data input unit 151. In other words, the external datacomparison unit 152 compares the face feature values of severalhigh-ranking image search results conducted by an external networkservice or the like from the external data input unit 151 withrepresentative face feature values of several high-ranking face groupsof small distance from the name/face group distance acquisition unit133.

In step S157, as a result of the comparison, the external datacomparison unit 152 treats the face group with the highest degree ofsimilarity with the external data as a first candidate face group, andsupplies information about several high-ranking face groups to theselection screen display control unit 153 and the face groupconfiguration unit 154.

In addition, in the case of determining to not use external data in stepS155, the process skips steps S156 and S157, and proceeds to step S158.

In step S158, in one process, the selection screen display control unit153 determines whether or not there is a user selection. In the case ofdetermining that there is a user selection in step S158, the processproceeds to step S159.

In step S159, the selection screen display control unit 153 generates aselection screen enabling the selection of one face group name fromamong candidate face group names made up of face group names that arecandidates for the face group to associate with the name. The selectionscreen display control unit 153 causes the display unit 141 to displaythe generated selection screen.

Note that in one process, when external data is used and there is a userselection, the selection screen display control unit 153 generates aselection screen made up of face group names and the like that arecandidates for the face group to associate with the name, on the basisof the face group information from the external data comparison unit152. In one process, when external data is not used and there is a userselection, the selection screen display control unit 153 generates aselection screen made up of face group names and the like that arecandidates for the face group to associate with the name, on the basisof the face group information from the name/face group distanceacquisition unit 133.

The user operates the operating input unit 142 to select a face groupname to associate with the name. The operating input unit 142 supplies aselection signal corresponding to the selection to the selection screendisplay control unit 153.

In step S160, the selection screen display control unit 153 acquires aselection result from the operating input unit 142. In step S161, theselection screen display control unit 153 treats the user-selected facegroup as a first candidate face group, and supplies information aboutseveral high-ranking face groups to the face group configuration unit154.

In step S158, in the case of determining that there is no userselection, the process skips steps S159 to S161, and proceeds to stepS162.

In step S162, the face group configuration unit 154 configures a facegroup. In other words, in one process, when there is a user selection,the face group configuration unit 154 configures the first candidateface group supplied from the selection screen display control unit 153as the face group to associate with the specific name. In one process,when external data is used and there is no user selection, the facegroup configuration unit 154 configures the first candidate face groupsupplied from the external data comparison unit 152 as the face group toassociate with the specific name. In one process, when external data isnot used and there is no user selection, the face group configurationunit 154 configures the first candidate face group supplied from thename/face group distance acquisition unit 133 as the face group toassociate with the specific name.

As above, in the information processing device 11 of FIG. 1, a nameoccurrence pattern indicating whether or not a user-specified nameoccurs within selected content is acquired, and face group occurrencepatterns indicating occurrence or non-occurrence in all selected contentare acquired. Subsequently, on the basis of the similarity between thename occurrence pattern and the face group occurrence patterns, a facegroup to associate with the user-specified name is determined.Consequently, more efficient name registration work is possible.

2. Embodiment Information Processing Device Another Configuration ofInformation Processing Device According to Present Technology

FIG. 16 is a diagram illustrating an exemplary configuration of aninformation processing device applying the present technology.

As an example, as in the information processing device 11 in FIG. 1, theinformation processing device 211 in FIG. 16 registers in a catalog aface image and a face image feature value corresponding to a name, andconducts searches on the catalog, even if the input name is notregistered in the catalog. As in the information processing device 11,the information processing device 211 is made up of a personal computeror the like, for example.

In the example of FIG. 16, the information processing device 211includes the name input unit 21, the content archive 22, a contentselection unit 221, a face group acquisition unit 222, a face groupdetermination unit 223, the registration face image selection unit 26,and the face catalog 27.

The information processing device 211 is equipped with a name input unit21, a content archive 22, a registration face image selection unit 26,and a face catalog 27, in common with the information processing device11 of FIG. 1. The information processing device 211 differs from theinformation processing device 11 of FIG. 1 in that the content selectionunit 23, the face group acquisition unit 24, and the face groupdetermination unit 25 are replaced with a content selection unit 221, aface group acquisition unit 222, and a face group determination unit223, respectively.

In other words, the content selection unit 221 selects, from inside thecontent archive 22, content in which the name specified from the nameinput unit 21 or the specific person possibly occur. Herein, theselected content is taken to be content within a specified range insidethe content archive 22. For example, the target of selection may belimited by the category of picture, capture time, capture location, orthe like.

The content selection unit 221 supplies information about the selectedcontent to the face group acquisition unit 222.

The face group acquisition unit 222 conducts a process that groups facesoccurring within each piece of content selected by the content selectionunit 221, and collects the same person into a single group.Subsequently, the face group acquisition unit 222 conducts a process ofacquiring, as candidate face groups, face groups with a high likelihoodof the specific person from among all face groups in all selectedcontent. Specifically, the face group acquisition unit 222 acquires,from among the face groups of the selected content, as candidate facegroups, face groups that have the greatest number of similar face groupsappearing in other content. The face group acquisition unit 222 suppliesinformation about the acquired candidate face groups to the face groupdetermination unit 223.

The face group determination unit 223 conducts a process of determininga face group to associate with the specific person from among thecandidate face groups acquired by the face group acquisition unit 222.Specifically, the face group determination unit 223 clusters thecandidate face groups, and determines the face group belonging to thecluster with the most candidate face groups as the face group toassociate with the name. The face group determination unit 223 suppliesinformation about the determined face group to the registration faceimage selection unit 26.

Operation of Information Processing Device

Next, a face catalog registration process which is an operation of theinformation processing device 211 will be described with reference tothe flowchart in FIG. 17.

For example, a name specified by the user is input into the contentselection unit 221 via the name input unit 21. In step S211, the contentselection unit 221 conducts a content selection process. The contentselection process will be described later in detail with reference toFIG. 18.

According to the process in step S211, content in which the specificperson possibly occurs is selected from inside the content archive 22,and supplied together with the content to the face group acquisitionunit 222.

In step S212, the face group acquisition unit 222 conducts a face groupacquisition process. The face group acquisition process will bedescribed later in detail with reference to FIG. 21.

According to the process in step S212, face images occurring in eachpiece of selected content are detected and grouped per person, and onlythe face groups that possibly correspond to the specific person areacquired as candidate face groups and supplied to the face groupdetermination unit 223.

In step S213, the face group determination unit 223 conducts a facegroup determination process. The face group determination process willbe described later in detail with reference to FIG. 26.

According to the process in step S213, a face group to associate withthe specific name is determined from among the collection of candidateface groups acquired by the face group acquisition unit 222. Informationabout the determined face group is supplied to the registration faceimage selection unit 26.

In step S214, the registration face image selection unit 26 conducts aregistration face image selection process. In other words, theregistration face image selection unit 26 selects a specified number offace images and face image feature values from among the face groupdetermined by the face group determination unit 25, and registers in theface catalog 27 the selected face images and face image feature valuesin association with the name.

According to the above, in the information processing device 211, faceimages and face image feature values are registered in the face catalog27 in association with a name, enabling searches to be conducted.Consequently, more efficient catalog registration work is possible.

Example of Content Selection Process

Next, the content selection process in step S211 of FIG. 17 will bedescribed with reference to the flowchart in FIG. 18. In the contentselection process, content in which the specific person possibly occursis acquired from inside the content archive 22.

The content selection unit 221 selects content from the content archive22 in step S231, and selects a frame in step S232.

In step S233, the content selection unit 221 conducts variousidentification processes on the selected frame. Note that in step S233,processes basically similar to the identification processes conducted instep S33 of FIG. 4 are conducted. In other words, in steps S232 andS233, at an arbitrary time interval, metadata attached to the content isread, and the various identification processes judge whether or not thespecific name or speech by the specific person occurs.

Herein, an identification process refers to a process of identifyingtext information in a screen, and may be a process of identifying spokencontent, and additionally is a process using text, speech, or otherinformation in a picture, such as a process of identifying a speaker.Consequently, the identification process is not required to be limitedto the above, insofar as it is possible to acquire text information oraudio information related to the specific name and identify whether ornot the specific person is present.

In step S234, every time the above identification processing isconducted, the content selection unit 221 updates specific personoccurrence frequency data as a result. For example, the specific personoccurrence frequency data is structured as discussed earlier withreference to FIG. 5. The details written to the specific personoccurrence frequency data are a time position at which identificationwas conducted, and information indicating whether or not there is anoccurrence. Herein, the information indicating whether or not there isan occurrence may be included for every identification processingresult, collected into a single piece of information, or both. Inaddition, information such as the size of the text and the volume ofspeech at the time of identification may also be written.

In step S235, the content selection unit 221 determines whether or notall frames have been processed. In the case of determining that not allframes have been processed in step S235, the process returns to stepS232, and the processing thereafter is repeated.

In step S235, in the case of determining that all frames have beenprocessed, the process proceeds to step S236. In step S236, the contentselection unit 221 determines whether or not the occurrence frequency isgreater than a threshold value, on the basis of the updated specificperson occurrence frequency data.

In step S236, in the case of determining that the occurrence frequencyis greater than the threshold value, the specific person is treated asoccurring, and the process proceeds to step S237. In step S237, thecontent selection unit 221 adds the content selected in step S231 to aspecific person occurrence content list.

In step S236, in the case of determining that the occurrence frequencyis less than the threshold value, the operation in step S237 is skipped,and the process proceeds to step S238.

Note that in step S236 discussed above, the determination of whether ornot the specific person occurs may be made with not only the occurrencefrequency, but also in combination with factors such as the size of thetext and the volume of speech at the time of identification.

In step S238, the content selection unit 221 determines whether or notthe above process has been conducted on all content. In the case ofdetermining that not all content has been processed in step S238, theprocess returns to step S231, and the processing thereafter is repeated.In the case of determining that all content has been processed in stepS238, the content selection process of FIG. 18 ends. Subsequently,information about the content in the specific person occurrence contentlist is supplied to the face group acquisition unit 222 as informationabout the selected content.

Note that an example is described in which the identification processingin the content selection process described above is conducted every timea name is specified. However, the identification processing is notrequired to be conducted every time a name is specified. In other words,information related to a person occurring in a piece of content and thefrequency of occurrence may be acquired in advance for each piece ofcontent and stored in memory or the like, for example, and on the basisof the information, the name occurrence pattern may be updated. Anexample of the content selection process in this case is illustrated inthe following FIG. 19.

Another Example of Content Selection Process

Next, another example of the content selection process in step S211 ofFIG. 17 will be described with reference to the flowchart in FIG. 19.

In step S251, the content selection unit 221 selects one piece ofcontent from the content archive 22. In step S252, the content selectionunit 23 loads name occurrence frequency data for the acquired contentfrom memory (not illustrated) or the like, for example. For example, theperson occurrence frequency data is structured as discussed earlier withreference to FIG. 7.

In step S253, the content selection unit 23 determines whether or notthe specific person occurs. For example, similarly to step S236 of FIG.18, by determining whether or not the occurrence frequency is greaterthan a threshold value, it is determined whether or not the specificperson occurs.

In step S253, in the case of determining that the occurrence frequencyis greater than the threshold value, or in other words, that thespecific person occurs, the process proceeds to step S254. In step S254,the content selection unit 23 adds the content selected in step S251 toa specific person occurrence content list.

In step S253, in the case of determining that the occurrence frequencyis less than the threshold value, or in other words, that the specificperson does not occur, the process skips step S254 and proceeds to stepS255.

In step S255, the content selection unit 221 determines whether or notthe above process has been conducted on all content. In the case ofdetermining that not all content has been processed in step S255, theprocess returns to step S251, and the processing thereafter is repeated.In the case of determining that all content has been processed in stepS255, the content selection process of FIG. 19 ends. Subsequently,information about the content in the specific person occurrence contentlist is supplied to the face group acquisition unit 222 as informationabout the selected content.

Exemplary Configuration of Face Group Acquisition Unit

FIG. 20 illustrates an exemplary configuration of a face groupacquisition unit.

In the example of FIG. 20, the face group acquisition unit 222 includesan in-content face group acquisition unit 271 and a candidate face groupacquisition unit 272.

The in-content face group acquisition unit 271 detects face imagesoccurring in each piece of content selected by the content selectionunit 221 (in other words, each piece of content in the specific personoccurrence content list), and groups the face images per person.Subsequently, the in-content face group acquisition unit 271 acquiresonly face groups that possibly correspond to the specific person. Inother words, face groups that are not the specific person are excluded.The in-content face group acquisition unit 271 supplies informationabout the acquired face groups to the candidate face group acquisitionunit 272.

The candidate face group acquisition unit 272 acquires candidates forthe face group to associate with the specific person (hereinafterdesignated candidate face groups) for each piece of content, andsupplies information about the acquired candidate face groups to theface group determination unit 223.

Example of Face Group Acquisition Process

Next, the face group acquisition process in step S212 of FIG. 17 will bedescribed with reference to the flowchart in FIG. 21.

In step S271, the in-content face group acquisition unit 271 conducts anin-content face group acquisition process. The in-content face groupacquisition process will be described later in detail with reference toFIG. 22.

According to the process in step S271, face images occurring in eachpiece of content selected by the content selection unit 221 are detectedand grouped per person, and only face groups that possibly correspond tothe specific person are acquired.

In step S272, the in-content face group acquisition unit 271 determineswhether or not the process of step S271 has been conducted on allcontent. In step S272, in the case of determining that not all contenthas been processed, the process returns to step S271, and the processingthereafter is repeated.

In step S272, in the case of determining that all content has beenprocessed, the process proceeds to step S273. In step S273, thecandidate face group acquisition unit 272 conducts a candidate facegroup acquisition process. The candidate face group acquisition processwill be described later with reference to FIG. 24.

According to the process of step S273, candidates for the face group toassociate with the specific person are acquired for each piece ofcontent, and information about the acquired candidate face groups issupplied to the face group determination unit 223. Subsequently, theface group acquisition process of FIG. 21 ends, and the process returnsto step S212 of FIG. 17.

Example of In-Content Face Group Acquisition Process

Next, the in-content face group acquisition process in step S271 of FIG.21 will be described with reference to the flowchart in FIG. 22.

In step S291, the in-content face group acquisition unit 271 conducts aface grouping process. Note that since the face grouping process isbasically similar to the face grouping process discussed earlier withreference to FIG. 10, further description thereof would be a repetition,and is thus omitted.

According to the process in step S291, face images occurring within thecontent are detected, the detected face images are grouped per person,and the face groups are registered in a face group list. Note that thisprocess is conducted on all content selected by the content selectionunit 221.

In step S292, the in-content face group acquisition unit 271 conductsscene division. In other words, the in-content face group acquisitionunit 271 divides the content at scene breaks.

In step S293, the in-content face group acquisition unit 271 configuresone scene interval from among the divided scene intervals, and in stepS294, determines whether or not a face appears in the configured sceneinterval. In step S294, in the case of determining that a face appears,the process proceeds to step S295.

In step S295, the in-content face group acquisition unit 271 loads thespecific person occurrence frequency data of the scene intervalconfigured in step S293. In other words, data of the relevant sceneinterval is loaded from among the specific person occurrence frequencydata acquired by the content selection unit 221.

In step S296, it is determined whether or not the specific person doesnot occur in a nearby scene including the relevant scene interval. Instep S296, in the case of determining that the specific person does notoccur in a nearby scene including the relevant scene interval, theprocess proceeds to step S297.

In this case, since there is an extremely low likelihood that a faceoccurring in the relevant scene is the specific person, in step S297,the in-content face group acquisition unit 271 removes the faceoccurring in the relevant scene interval from the face group list.Consequently, face images occurring in a scene in which the specificperson does not occur may be removed from the candidates.

On the other hand, in step S296, in the case of determining that thespecific person occurs in a nearby scene including the relevant sceneinterval, there is a high likelihood that a face occurring in therelevant scene is the specific person, and thus the process skips stepS297 and proceeds to step S298.

In addition, in step S294, in the case of determining that a face doesnot appear, the process skips steps S295 to S297, and proceeds to stepS298.

In step S298, the in-content face group acquisition unit 271 determineswhether or not the above process has been conducted on all scenes. Instep S298, in the case of determining that the above process has notbeen conducted on all scenes, the process returns to step S293, and theprocessing thereafter is repeated.

In step S298, in the case of determining that the above process has beenconducted on all scenes, the process proceeds to step S299.

In step S299, the in-content face group acquisition unit 271 selects oneface group from the face group list, and conducts personalidentification per face group. Subsequently, in step S300, thein-content face group acquisition unit 271 determines whether or not aface belonging to that face group matches the face of a person alreadyregistered in the catalog.

In step S300, in the case of determining that a face belonging to theface group matches the face of a person already registered in thecatalog, the process proceeds to step S301. In this case, since the facegroup is clearly not of a face that should be associated with thespecific person input from the name input unit 21, in step S301, thein-content face group acquisition unit 271 removes that face group fromthe face group list.

In addition, in step S300, in the case of determining that a facebelonging to the face group does not match the face of a person alreadyregistered in the catalog, the process proceeds to step S302. In otherwords, in this case, the in-content face group acquisition unit 271keeps the face group in the face group list, and in step S302, acquiresa representative face image within the face group. The representativeface image is taken to be an average face of the face feature values forall faces within the face group.

After that, in step S303, the in-content face group acquisition unit 271determines whether or not the above process has been conducted on allface groups. In step S303, in the case of determining that the aboveprocess has not been conducted on all face groups, the process returnsto step S299, and the processing thereafter is repeated.

In step S303, in the case of determining that the above process has beenconducted on all face groups, the in-content face group acquisitionprocess of FIG. 22 ends, and the process returns to step S271 in FIG.21.

As above, within each piece of content, face groups are created andregistered in a face group list. Subsequently, groups of face imagesthat occur in scenes in which the specific person does not occur andface groups that are not of a face that should be associated with thespecific person are removed from the face group list. Subsequently, as aresult, within each piece of content, a face group list made up of facegroups that should be associated with the specific person is acquired.

Example of Candidate Face Group Acquisition Process

Next, the candidate face group acquisition process in step S273 of FIG.21 will be described with reference to the flowchart in FIG. 23. Notethat the example of FIG. 23 will be described with reference to FIG. 24where appropriate.

For example, as illustrated in FIG. 24, content A to C is acquired ascontent in which the specific person occurs, and as a result of facegrouping for each piece of content, in the content A, face groups A1 toA3 are acquired. In the content B, face groups B1 to B3 are acquired.Also, in the content C, face groups C1 and C2 are acquired.

In step S331, the candidate face group acquisition unit 272 selects apiece of content (for example, the content A). In step S332, thecandidate face group acquisition unit 272 selects a face group (forexample, the face group A1) from the face group list of the selectedcontent.

In step S333, the candidate face group acquisition unit 272 acquires,from the face group lists of other pieces of content, the number ofsimilar face groups appearing in the other pieces of content. Note thatthe representative image of each face group may also be used to computethe similarity between face groups. For example, similarity is evaluatedbetween the face group A1 of the content A, and each of the face groupsB1 to B3, C1, and C2 belonging to the content B and C other than thecontent A, and the number of times the similarity meets or exceeds athreshold value is acquired as a similarity group count N_(A1).

In step S334, the candidate face group acquisition unit 272 determineswhether or not all face groups in the selected piece of content havebeen processed. In step S334, in the case of determining that not allface groups have been processed, the process returns to step S332, andthe processing thereafter is repeated.

In other words, a similar process is conducted on the other face groupsA2 and A3 other than the content A, and similar face group counts N_(A2)and N_(A3) are acquired.

In step S334, in the case of determining that all face groups have beenprocessed, the process proceeds to step S335. In step S335, thecandidate face group acquisition unit 272 adds the face group with thegreatest similarity face group count to a candidate face group.

In other words, the face group having the maximum value from among allsimilar face group counts N_(A1) to N_(A3) is acquired as a candidateface group. At this point, if there are multiple maximum values of thesimilar face group count, the multiple face groups having the maximumvalue are acquired as candidate face groups.

In step S336, the candidate face group acquisition unit 272 determineswhether or not all content has been processed. In step S336, in the caseof determining that not all content has been processed, the processreturns to step S331, and the processing thereafter is repeated.

In other words, the above process is also conducted on the content B andC, and candidate face groups are acquired from all pieces of content.

On the other hand, in step S336, in the case of determining that theabove process has been conducted on all content, the candidate facegroup acquisition process of FIG. 23 ends, and the process returns tostep S273 in FIG. 21.

As above, for every face group in each piece of content, a count of thenumber of similar face groups appearing in other content is acquired,and the face group with the greatest acquired similarity face groupcount is added to a candidate face group.

Exemplary Configuration of Face Group Determination Unit

FIG. 25 illustrates an exemplary configuration of a face groupacquisition unit.

In the example of FIG. 25, the face group determination unit 223includes a candidate face group input unit 411, a clustering unit 412, aselection screen display control unit 413, and a face groupconfiguration unit 414.

Also, in this case, the information processing device 11 also includesthe display unit 141 and the operating input unit 142 discussed withreference to FIG. 12 in addition to the configuration discussed earlierwith reference to FIG. 16.

The candidate face group input unit 411 inputs and supplies to theclustering unit 412 a candidate face group of each piece of content fromthe candidate face group acquisition unit 272.

The clustering unit 412 conducts clustering using the face featurevalues of the representative face image of each candidate face group,and collects the faces of the same person into a single cluster.

For the clustering, a method is used in which clusters are joined untilthe distances between all clusters in hierarchical clustering becomegreater than a threshold value used to determine the same face.Alternatively, for the clustering, a method is used in which the clusteris partitioned into two clusters by partition-optimization clusteringsuch as k-means, and cluster partitioning is repeated until the spreadof the clusters becomes less than a threshold value. The clusteringmethod is not limited to these methods.

As a result of the clustering, the clustering unit 412 treats the facegroup belonging to the cluster with the greatest number of face groupsconstituting the cluster as a first candidate face group, and suppliesinformation about several high-ranking face groups to the selectionscreen display control unit 413 and the face group configuration unit414.

The selection screen display control unit 413, in one process, whenthere is a user selection, generates a selection screen enabling theselection of one face group name from among candidate face group namesmade up of face group names that are candidates for the face group toassociate with the specific person. The selection screen display controlunit 413 causes the display unit 141 to display the generated selectionscreen. In addition, on the basis of a face group selection signal bythe user input via the operating input unit 142, the selection screendisplay control unit 413 treats the user-selected face group as a firstcandidate face group, and supplies information about severalhigh-ranking face groups to the face group configuration unit 414.

In one process, when there is a user selection, the face groupconfiguration unit 414 configures the first candidate face groupsupplied from the selection screen display control unit 413 as the facegroup to associate with the specific name. In one process, when there isno user selection, the face group configuration unit 414 configures thefirst candidate face group supplied from the clustering unit 412 as theface group to associate with the specific name.

Example of Face Group Determination Process

Next, the face group determination process in step S213 of FIG. 17executed by the face group determination unit 223 of FIG. 25 will bedescribed with reference to the flowchart in FIG. 26.

In step S411, the candidate face group input unit 411 inputs andsupplies to the clustering unit 412 the candidate face groups from thecandidate face group acquisition unit 272.

In step S412, the clustering unit 412 conducts clustering using the facefeature values of the representative face image of each candidate facegroup, and collects the faces of the same person into a single cluster.

In step S413, as a result of the clustering, the clustering unit 412treats the face group belonging to the cluster with the greatest numberof face groups constituting the cluster as a first candidate face group,and supplies information about several high-ranking face groups to theselection screen display control unit 413 and the face groupconfiguration unit 414.

In step S414, in one process, the selection screen display control unit413 determines whether or not there is a user selection. In the case ofdetermining that there is a user selection in step S414, the processproceeds to step S415.

In step S415, the selection screen display control unit 413 generates aselection screen enabling the selection of one face group name fromamong candidate face group names made up of face group names that arecandidates for the face group to associate with the specific person. Theselection screen display control unit 413 causes the display unit 141 todisplay the generated selection screen.

The user operates the operating input unit 142 to select a face groupname to associate with the specific person. The operating input unit 142supplies a selection signal corresponding to the selection to theselection screen display control unit 413.

In step S416, the selection screen display control unit 413 acquires aselection result from the operating input unit 142. In step S417, theselection screen display control unit 413 treats the user-selected facegroup as a first candidate face group, and supplies information aboutseveral high-ranking face groups to the face group configuration unit414.

In step S414, in the case of determining that there is no userselection, the process skips steps S415 to S417, and proceeds to stepS418.

In step S418, the face group configuration unit 414 configures a facegroup. In other words, in one process, when there is a user selection,the face group configuration unit 414 configures the first candidateface group supplied from the selection screen display control unit 413as the face group to associate with the specific name. In one process,when there is no user selection, the face group configuration unit 414configures the first candidate face group supplied from the clusteringunit 412 as the face group to associate with the specific name.

Note that although the above describes an example of determining a facegroup in which several high-ranking face groups are treated ascandidates and presented to the user for selection by the user, asdescribed next, a selection may also be made using a network service orthe like as external data.

Exemplary Configuration of Face Group Determination Unit

FIG. 27 is a block diagram illustrating an exemplary configuration of aface group determination unit that makes a selection using a networkservice or the like as external data.

In the example of FIG. 27, the face group determination unit 223includes the candidate face group input unit 411 and the clustering unit412. Additionally, the face group determination unit 223 includes anexternal data input unit 431, an external data comparison unit 432, aselection screen display control unit 433, and a face groupconfiguration unit 434.

The face group determination unit 223 of FIG. 27 is equipped with acandidate face group input unit 411 and a clustering unit 412, in commonwith the face group determination unit 223 of FIG. 25.

The face group determination unit 223 of FIG. 27 differs from the facegroup determination unit 223 of FIG. 25 in that the selection screendisplay control unit 413 and the face group configuration unit 414 arereplaced with the selection screen display control unit 433 and the facegroup configuration unit 434. In addition, the face group determinationunit 223 of FIG. 27 differs from the face group determination unit 223of FIG. 25 with the addition of the external data input unit 431 and theexternal data comparison unit 432.

In other words, clustering unit 412 treats the face group belonging tothe most clusters as a first candidate face group, and suppliesinformation about several high-ranking face groups to the external datacomparison unit 432, the selection screen display control unit 433, andthe face group configuration unit 434.

The external data input unit 431 inputs and supplies to the externaldata comparison unit 432 external data made up of face feature valuesand the like of several high-ranking image search results conducted byan external network service or the like for the same name as the oneinput into the name input unit 21 of FIG. 16.

The external data comparison unit 432 compares the face feature valuesof several high-ranking image search results conducted by an externalnetwork service or the like from the external data input unit 431 withrepresentative face feature values of several high-ranking face groupsof small distance from the name/face group distance acquisition unit133. As a result of the comparison, the external data comparison unit432 treats the face group with the highest degree of similarity with theexternal data as a first candidate face group, and supplies informationabout several high-ranking face groups to the selection screen displaycontrol unit 433 and the face group configuration unit 434.

In one process, when external data is used and there is a userselection, the selection screen display control unit 433 generates aselection screen made up of face group names and the like that arecandidates for the face group to associate with the specific person, onthe basis of the face group information from the external datacomparison unit 432. In one process, when external data is not used andthere is a user selection, the selection screen display control unit 433generates a selection screen made up of face group names and the likethat are candidates for the face group to associate with the specificperson, on the basis of the face group information from the clusteringunit 412. The selection screen display control unit 413 causes thedisplay unit 141 to display the generated selection screen.

On the basis of a face group selection signal by the user input via theoperating input unit 142, the selection screen display control unit 413treats the user-selected face group as a first candidate face group, andsupplies information about several high-ranking face groups to the facegroup configuration unit 434.

In one process, when there is a user selection, the face groupconfiguration unit 434 configures the first candidate face groupsupplied from the selection screen display control unit 433 as the facegroup to associate with the specific name. In one process, when externaldata is used and there is no user selection, the face groupconfiguration unit 434 configures the first candidate face groupsupplied from the external data comparison unit 432 as the face group toassociate with the specific name. In one process, when external data isnot used and there is no user selection, the face group configurationunit 434 configures the first candidate face group supplied from theclustering unit 412 as the face group to associate with the specificname.

Another Example of Face Group Determination Process

Next, the face group determination process in step S213 of FIG. 17executed by the face group determination unit 223 of FIG. 27 will bedescribed with reference to the flowchart in FIG. 28.

In step S431, the candidate face group input unit 411 inputs andsupplies to the clustering unit 412 the candidate face groups from thecandidate face group acquisition unit 272.

In step S432, the clustering unit 412 conducts clustering using the facefeature values of the representative face image of each candidate facegroup, and collects the faces of the same person into a single cluster.

In step S433, as a result of the clustering, the clustering unit 412treats the face group belonging to the cluster with the greatest numberof face groups constituting the cluster as a first candidate face group,and supplies information about several high-ranking face groups to theselection screen display control unit 433 and the face groupconfiguration unit 434.

In step S434, in one process, the external data comparison unit 432determines whether or not to use external data. In the case ofdetermining to use external data in step S434, the process proceeds tostep S435.

In step S435, the external data comparison unit 432 conducts a processof comparing several high-ranking face groups belonging to the mostclusters from the clustering unit 412 with external data from theexternal data input unit 431. In other words, the external datacomparison unit 432 compares the face feature values of severalhigh-ranking image search results conducted by an external networkservice or the like from the external data input unit 431 withrepresentative face feature values of several high-ranking face groupsbelonging to the most clusters from the clustering unit 412.

In step S436, as a result of the comparison, the external datacomparison unit 432 treats the face group with the highest degree ofsimilarity with the external data as a first candidate face group, andsupplies information about several high-ranking face groups to theselection screen display control unit 433 and the face groupconfiguration unit 434.

In in the case of determining to not use external data in step S434, theprocess skips steps S435 and S436, and proceeds to step S437.

In step S437, in one process, the selection screen display control unit433 determines whether or not there is a user selection. In the case ofdetermining that there is a user selection in step S437, the processproceeds to step S438.

In step S438, the selection screen display control unit 433 generates aselection screen made up of information such as face group names thatare candidates for the face group to associate with the name, and causesthe display unit 141 to display the generated selection screen.

Note that in one process, when external data is used and there is a userselection, the selection screen display control unit 433 generates aselection screen made up of face group names and the like that arecandidates for the face group to associate with the name, on the basisof the face group information from the external data comparison unit432. In one process, when external data is not used and there is a userselection, the selection screen display control unit 433 generates aselection screen made up of face group names and the like that arecandidates for the face group to associate with the specific person, onthe basis of the face group information from the clustering unit 412.

The user operates the operating input unit 142 to select a face groupname to associate with the name. The operating input unit 142 supplies aselection signal corresponding to the selection to the selection screendisplay control unit 433.

In step S439, the selection screen display control unit 433 acquires aselection result from the operating input unit 142. In step S440, theselection screen display control unit 433 treats the user-selected facegroup as a first candidate face group, and supplies information aboutseveral high-ranking face groups to the face group configuration unit434.

In step S437, in the case of determining that there is no userselection, the process skips steps S438 to S440, and proceeds to stepS441.

In step S441, the face group configuration unit 434 configures a facegroup. In other words, in one process, when there is a user selection,the face group configuration unit 434 configures the first candidateface group supplied from the selection screen display control unit 433as the face group to associate with the specific name. In one process,when external data is used and there is no user selection, the facegroup configuration unit 434 configures the first candidate face groupsupplied from the external data comparison unit 432 as the face group toassociate with the specific name. In one process, when external data isnot used and there is no user selection, the face group configurationunit 434 configures the first candidate face group supplied from theclustering unit 412 as the face group to associate with the specificname.

As above, in the information processing device 211 of FIG. 16, contentin which a name possibly occurs is selected, and from among face groupsin each piece of selected content, the face groups having the greatestnumber of similar face groups appearing in other content are acquired ascandidate face groups. Subsequently, the candidate face groups areclustered, and the face group belonging to the cluster with the mostcandidate face groups is determined as the face group to associate withthe name. Consequently, more efficient name registration work ispossible.

As above, according to the present technology, since a face image and aface image feature value corresponding to a name are registered in acatalog, searches may be conducted, even if the input name is notregistered in the catalog, for example.

In other words, in a name-based video or image search of the past, aspecific person is determined to occur or not according to a comparisonwith the face feature values of persons already registered in a catalog,and videos or images in which the specific person occurs are presentedas search results.

According to the present technology, the registration of a name and aface image feature value into the catalog is automated, thereby enablingmore efficient work.

Particularly, for the association of a person and a face image, it ispossible to conduct association on the basis of not only manuallyapplied information that may be accurate, such as an EPG, for example,but also information related to names automatically acquired by variousidentification processes from within a video.

In addition, according to the present technology, it is possible toassociate a name with a face image having a relatively high occurrencefrequency rather than a face image common within all content.Consequently, the association of a name and a face image becomespossible even in the hypothetical case in which the specific person doesnot occur or a face image of the person could not be detected in thecandidates of moving images in which the person occurs.

Furthermore, according to the present technology, during catalogregistration, a representative face of each cluster obtained byclustering selected face image collections is selected. Consequently, aface image having various features may be registered, and personalidentification performance may be improved.

The series of processes described above can be executed by hardware butcan also be executed by software. When the series of processes isexecuted by software, a program that constructs such software isinstalled into a computer. Here, the expression “computer” includes acomputer in which dedicated hardware is incorporated and ageneral-purpose personal computer or the like that is capable ofexecuting various functions when various programs are installed.

3. Third Embodiment Computer Configuration Example of Computer

FIG. 29 illustrates a configuration example of hardware of a computerthat executes the above series of processes by programs.

In the computer 500, a central processing unit (CPU) 501, a read onlymemory (ROM) 502 and a random access memory (RAM) 503 are mutuallyconnected by a bus 504.

An input/output interface 505 is also connected to the bus 504. An inputunit 506, an output unit 507, a storage unit 508, a communication unit509, and a drive 510 are connected to the input/output interface 505.

The input unit 506 is configured from a keyboard, a mouse, a microphoneor the like. The output unit 507 configured from a display, a speaker orthe like. The storage unit 508 is configured from a hard disk, anon-volatile memory or the like. The communication unit 509 isconfigured from a network interface or the like. The drive 510 drives aremovable recording medium 511 such as a magnetic disk, an optical disk,a magneto-optical disk, a semiconductor memory or the like.

In the computer configured as described above, the CPU 501 loads aprogram that is stored, for example, in the storage unit 508 onto theRAM 503 via the input/output interface 505 and the bus 504, and executesthe program. Thus, the above-described series of processing isperformed.

As one example, the program executed by the computer (the CPU 501) maybe provided by being recorded on the removable recording medium 511 as apackaged medium or the like. The program can also be provided via awired or wireless transfer medium, such as a local area network, theInternet, or a digital satellite broadcast.

In the computer, by loading the removable recording medium 511 into thedrive 510, the program can be installed into the storage unit 508 viathe input/output interface 505. It is also possible to receive theprogram from a wired or wireless transfer medium using the communicationunit 509 and install the program into the storage unit 508. As anotheralternative, the program can be installed in advance into the ROM 502 orthe storage unit 508.

It should be noted that the program executed by a computer may be aprogram that is processed in time series according to the sequencedescribed in this specification or a program that is processed inparallel or at necessary timing such as upon calling.

In the present disclosure, steps of describing the above series ofprocesses may include processing performed in time-series according tothe description order and processing not processed in time-series butperformed in parallel or individually.

An embodiment of the disclosure is not limited to the embodimentsdescribed above, and various changes and modifications may be madewithout departing from the scope of the disclosure.

For example, the present disclosure can adopt a configuration of cloudcomputing which processes by allocating and connecting one function by aplurality of apparatuses through a network.

For example, the present disclosure can adopt a configuration of cloudcomputing which processes by allocating and connecting one function by aplurality of apparatuses through a network.

In addition, in the case where a plurality of processes is included inone step, the plurality of processes included in this one step can beexecuted by one apparatus or by allocating a plurality of apparatuses.

Further, an element described as a single device (or processing unit)above may be divided to be configured as a plurality of devices (orprocessing units). On the contrary, elements described as a plurality ofdevices (or processing units) above may be configured collectively as asingle device (or processing unit). Further, an element other than thosedescribed above may be added to each device (or processing unit).Furthermore, a part of an element of a given device (or processing unit)may be included in an element of another device (or another processingunit) as long as the configuration or operation of the system as a wholeis substantially the same. In other words, the present technology is notlimited to the embodiments described above, and various changes andmodifications may be made without departing from the scope of thetechnology.

The preferred embodiments of the present invention have been describedabove with reference to the accompanying drawings, whilst the presentinvention is not limited to the above examples, of course. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present invention.

Additionally, the present technology may also be configured as below.

(1)

An information processing device including:

a content selection unit configured to select content including at leastone name specified by a user;

a face group acquisition unit configured to acquire face groups bygrouping, per person, face images occurring in content selected by thecontent selection unit; and

a face group determination unit configured to determine a face group toassociate with the name from face groups acquired by the face groupacquisition unit.

(2)

The image processing device according to (1), wherein

the content selection unit acquires a name occurrence pattern indicatingwhether or not the name occurs within selected content,

the face group acquisition unit acquires face group occurrence patternsindicating whether or not there is an occurrence in all content selectedby the content selection unit, and

the face group determination unit determines a face group to associatewith the name on the basis of a similarity between the name occurrencepattern acquired by the content selection unit, and the face groupoccurrence patterns acquired by the face group acquisition unit.

(3)

The image processing device according to (2), wherein

the content selection unit acquires the name occurrence pattern on thebasis of text information or speech information within selected content,or specific person occurrence frequency data in which an occurrencefrequency of a specific person obtained as a result of identifyingmetadata attached to content is expressed in a time series.

(4)

The image processing device according to (2) or (3), further including:

a display control unit configured to control display of a screenenabling selection of one face group from a plurality of face groups forwhich the similarity is high; and

a selection unit configured to select the one face group according to auser operation, wherein

the face group determination unit determines a face group selected bythe selection unit as the face group to associate with the name.

(5)

The image processing device according to any one of (2) to (4), furtherincluding:

a comparison unit configured to compare a face group for which thesimilarity is high with external search result data for the name,wherein

the face group determination unit determines a face group that is mostsimilar to the external search result data from among a plurality offace groups for which the similarity is high as the face group toassociate with the name.

(6)

The image processing device according to (1), wherein

the content selection unit selects content in which the name possiblyoccurs,

the face group acquisition unit acquires, from among face groups in eachpiece of content selected by the content selection unit, as candidateface groups, face groups having a greatest number of similar face groupsappearing in other content, and

the face group determination unit clusters candidate face groupsacquired by the face group acquisition unit, and determines a face groupbelonging to a cluster with the most candidate face groups as the facegroup to associate with the name.

(7)

The image processing device according to (6), wherein

the content selection unit selects content in which the name possiblyoccurs on the basis of text information or speech information withinselected content, or specific person occurrence frequency data in whichan occurrence frequency of a specific person obtained as a result ofidentifying metadata attached to content is expressed in a time series.

(8)

The image processing device according to (6) or (7), wherein

the face group acquisition unit excludes face groups having a lowpossibility to associate with the name from among face groups in eachpiece of content selected by the content selection unit, and acquires,from among the other face groups, as candidate face groups, face groupshaving a greatest number of similar face groups appearing in othercontent.

(9)

The image processing device according to (6) or (7), further including:

a display control unit configured to control display of a screenenabling selection of one face group from a plurality of face groupsbelonging to clusters with many of the candidate face groups; and

a selection unit configured to select the one face group according to auser operation, wherein

the face group determination unit determines a face group selected bythe selection unit as the face group to associate with the name.

(10)

The image processing device according to (6) or (7), further including:

a comparison unit configured to compare a plurality of face groupsbelonging to clusters with many of the candidate face groups withexternal search result data for the name, wherein

the face group determination unit determines a face group that is mostsimilar to the external search result data from among a plurality offace groups belonging to clusters with many of the candidate face groupsas the face group to associate with the name.

(11)

The image processing device according to any one of (1) to (10), furtherincluding:

a face image selection unit configured to select a face image toregister in a catalog from among a face image collection in a face groupdetermined by the face group determination unit.

(12)

The image processing device according to (11), wherein

the face image selection unit selects a representative face in eachcluster as a result of clustering from among a face image collection ina face group determined by the face group determination unit as a faceimage to register in a catalog.

(13)

An information processing method performed by an information processingdevice, including:

selecting content including at least one name specified by a user;

acquiring face groups by grouping, per person, face images occurring inthe selected content; and

determining a face group to associate with the name from the acquiredface groups.

(14)

A program causing a computer to function as:

a content selection unit configured to select content including at leastone name specified by a user;

a face group acquisition unit configured to acquire face groups bygrouping, per person, face images occurring in content selected by thecontent selection unit; and

a face group determination unit configured to determine a face group toassociate with the name from face groups acquired by the face groupacquisition unit.

REFERENCE SIGNS LIST

-   11 information processing device-   21 name input unit-   22 content archive-   23 content selection unit-   24 face group acquisition unit-   25 face group determination unit-   26 registration face image selection unit-   27 face catalog-   131 name occurrence pattern input unit-   132 face group occurrence pattern input unit-   133 name/face group distance acquisition unit-   134 selection screen display control unit-   135 face group configuration unit-   141 display unit-   142 operating input unit-   151 external data input unit-   152 external data comparison unit-   153 selection screen display control unit-   154 face group configuration unit-   211 information processing device-   221 content selection unit-   222 face group acquisition unit-   223 face group determination unit-   271 in-content face group acquisition unit-   272 candidate face group acquisition unit-   411 candidate face group input unit-   412 clustering unit-   413 selection screen display control unit-   414 face group configuration unit-   431 external data input unit-   432 external data comparison unit-   433 selection screen display control unit-   434 face group configuration unit

1. An information processing device comprising: a content selection unitconfigured to select content including at least one name specified by auser; a face group acquisition unit configured to acquire face groups bygrouping, per person, face images occurring in content selected by thecontent selection unit; and a face group determination unit configuredto determine a face group to associate with the name from face groupsacquired by the face group acquisition unit.
 2. The informationprocessing device according to claim 1, wherein the content selectionunit acquires a name occurrence pattern indicating whether or not thename occurs within selected content, the face group acquisition unitacquires face group occurrence patterns indicating whether or not thereis an occurrence in all content selected by the content selection unit,and the face group determination unit determines a face group toassociate with the name on the basis of a similarity between the nameoccurrence pattern acquired by the content selection unit, and the facegroup occurrence patterns acquired by the face group acquisition unit.3. The information processing device according to claim 2, wherein thecontent selection unit acquires the name occurrence pattern on the basisof text information or speech information within selected content, orperson occurrence frequency data in which an occurrence frequency of aperson obtained as a result of identifying metadata attached to contentis expressed in a time series.
 4. The information processing deviceaccording to claim 2, further comprising: a display control unitconfigured to control display of a screen enabling selection of one facegroup from a plurality of face groups for which the similarity is high;and a selection unit configured to select the one face group accordingto a user operation, wherein the face group determination unitdetermines a face group selected by the selection unit as the face groupto associate with the name.
 5. The information processing deviceaccording to claim 2, further comprising: a comparison unit configuredto compare a face group for which the similarity is high with externalsearch result data for the name, wherein the face group determinationunit determines a face group that is most similar to the external searchresult data from among a plurality of face groups for which thesimilarity is high as the face group to associate with the name.
 6. Theinformation processing device according to claim 1, wherein the contentselection unit selects content in which the name possibly occurs, theface group acquisition unit acquires, from among face groups in eachpiece of content selected by the content selection unit, as candidateface groups, face groups having a greatest number of similar face groupsappearing in other content, and the face group determination unitclusters candidate face groups acquired by the face group acquisitionunit, and determines a face group belonging to a cluster with the mostcandidate face groups as the face group to associate with the name. 7.The information processing device according to claim 6, wherein thecontent selection unit selects content in which the name possibly occurson the basis of text information or speech information within selectedcontent, or person occurrence frequency data in which an occurrencefrequency of a person obtained as a result of identifying metadataattached to content is expressed in a time series.
 8. The informationprocessing device according to claim 6, wherein the face groupacquisition unit excludes face groups having a low possibility toassociate with the name from among face groups in each piece of contentselected by the content selection unit, and acquires, from among theother face groups, as candidate face groups, face groups having agreatest number of similar face groups appearing in other content. 9.The information processing device according to claim 6, furthercomprising: a display control unit configured to control display of ascreen enabling selection of one face group from a plurality of facegroups belonging to clusters with many of the candidate face groups; anda selection unit configured to select the one face group according to auser operation, wherein the face group determination unit determines aface group selected by the selection unit as the face group to associatewith the name.
 10. The information processing device according to claim6, further comprising: a comparison unit configured to compare aplurality of face groups belonging to clusters with many of thecandidate face groups with external search result data for the name,wherein the face group determination unit determines a face group thatis most similar to the external search result data from among aplurality of face groups belonging to clusters with many of thecandidate face groups as the face group to associate with the name. 11.The information processing device according to claim 1, furthercomprising: a face image selection unit configured to select a faceimage to register in a catalog from among a face image collection in aface group determined by the face group determination unit.
 12. Theinformation processing device according to claim 11, wherein the faceimage selection unit selects a representative face in each cluster as aresult of clustering from among a face image collection in a face groupdetermined by the face group determination unit as a face image toregister in a catalog.
 13. An information processing method performed byan information processing device, comprising: selecting contentincluding at least one name specified by a user; acquiring face groupsby grouping, per person, face images occurring in the selected content;and determining a face group to associate with the name from theacquired face groups.
 14. A program causing a computer to function as: acontent selection unit configured to select content including at leastone name specified by a user; a face group acquisition unit configuredto acquire face groups by grouping, per person, face images occurring incontent selected by the content selection unit; and a face groupdetermination unit configured to determine a face group to associatewith the name from face groups acquired by the face group acquisitionunit.